It’s that time of the month. Open Site Explorer, the Mozbar, our tools and the API all have new link data as of today.
Much of this index is quite fresh, but with a caveat – there was a period of crawler downtime so some data from early in the index may be older (histogram below). It’s slightly smaller than our prior index, coming in at 69 Billion URLs (compared to July 6th’s 72 Billion).
(note that some data may be as old as May, but anything re-crawled in July would overwrite that older data, so there should be a significant amount of very fresh stuff in this index)
And, in more potentially good news (though it’s hard to count chickens before they hatch given AWS’ unreliability), we have another index that may be ready in the next 2-3 weeks, meaning more consistent updates. We’re kicking off indices nearly every week at this point to help account for Amazon’s weaknesses and give us a better chance of delivering on-time updates. This has pushed our costs into stratospheric territory ($600-$700K+/month to run Mozscape these days), but we can tolerate this pain a little longer while we work to move to our new hybrid cloud on our own, far-more-stable machines.
Let’s take a look at the full metrics for this index:
- 69,615,071,013 (69 billion) URLs
- 692,061,972 (692 million) Subdomains
- 164,242,043 (164 million) Root Domains
- 712,648,940,825 (712 billion) Links
-
Followed vs. Nofollowed
- 2.26% of all links found were nofollowed
- 55.01% of nofollowed links are internal
- 44.99% are external
-
Rel Canonical
- 13.45% of all pages now employ a rel=canonical tag
-
The average page has 74 links on it
- 62.43 internal links on average
- 11.13 external links on average
And here are the latest correlations between Mozscape metrics and Google’s search results:
- Page Authority – 0.34
- Domain Authority – 0.23
- MozRank – 0.20
- Linking Root Domains – 0.24
- Total Links – 0.20
- External Links – 0.24
You’ll also find that some of the bugs around anchor text from the prior index are much improved. There may be a few artifacts of odd data, but 99% of queries won’t surface these.
Happy link data day to all! And please write in with feedback.
